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DETECTION OF MULTIPLE WATERMARKS AND IMPROVED 
WATERMARK CALIBRATION SIGNALS 

Related Application Data 
This application claims priority benefit to copending provisional application 
60/257,924, filed December 21, 2000. 

Field of the Invention 
The present invention relates to digital watermark technology, and particularly 
concerns situations in which several digitally watermarked objects are presented to a 
watermark detector. 

Background and Summary of the Invention 

Digital watermarking systems are known in the art, and are shown, e.g., in 
copending applications 09/503,881 and 09/452,023. 

Steganographic calibration signals (sometimes termed "orientation signals," 
"marker signals," reference signals," "grid signals," etc.) are sometimes included with 
digital watermarking signals so that subsequent distortion of the object thereby marked 
(e.g., a digital image file, audio clip, document, etc.) can later be discerned and 
compensated-for. Such arrangements are detailed in the cited applications. 

Systems for detecting watermarks from marked objects sometimes fail if the 
image object encompasses several separately-watermarked components (e.g., a scanned 
magazine page with several different images, or photocopy data resulting from scanning 
while several documents are on the photocopier platen, as shown in Fig. 1). Each 
component may include its own calibration signal, confusing the detection system. 

In accordance with certain embodiments of the present invention, this problem is 
addressed by a proximity-based approach, and/or a multiple grids-based approach. 
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In accordance with other embodiments of the present invention, the calibration 
signal can - itself - convey the watermark information, so it serves both a calibration and 
a payload-conveyance function. 

The foregoing and additional features and advantage of the present invention will 
be more readily apparent from the following detailed description, which proceeds with 
reference to the accompanying drawings. 

Brief Description of the Drawings 
Fig. 1 shows several documents as they may be positioned on the platen of a 
photocopier. 

Fig. 2 is a flow chart detailing one embodiment of a multiple-grid detection 
approach. 

Fig. 3 is a flow chart further detailing the embodiment of Fig. 2. 

Detailed Description 
For expository convenience, the following discussion focuses on the context of 
processing image data resulting from scanning several documents on the same 
photocopier platen. It should be recognized, however, that the invention is not so limited, 
but finds application, e.g., in contexts involving audio, video and other content objects. 

Proximity approach 

The proximity approach is based on selecting and clubbing together image blocks 
that are in close proximity to each other. For two watermarked documents in an image, 
this approach would ideally give two clusters of blocks, one for each watermark. This 
approach requires modification to the methodology described in the '881 application. 
The prior methodology calculated the variance and edges in each block and used these to 
compute a metric for the block. Blocks were ranked based on their metric, and the top M 
blocks are selected to perform grid detection. The proximity approach, in contrast, takes 
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into account the spatial locations of the blocks, and/or their distances to each other, for 
division into one or more clusters. 

Such a proximity approach has several advantages. These include: 

• Separate block clusters can be treated as independent sets of data for grid detection 
and further decoding. 

• Except for the block clustering and proximity determination, prior art techniques 
(e.g., as in the '881 application) can remain unchanged. 

Although the proximity-based approach overcomes many of the shortcomings of 
the prior art, it has attributes that may render it poorly suited for certain applications. For 
example: 

• The proximity approach may result in blocks from a single watermarked document 
being divided into more than one cluster. 

• The proximity approach may fail for multiple watermarked documents that are either 
overlapped or in close proximity to each other. 

• The extensibility of this approach to situations where the number of watermarked 
components is more than two or three is uncertain. 

These shortcomings are generally obviated by the multiple grids approach. 

Multiple grids approach 

The multiple grids approach tackles the problem by searching for multiple grids 
during grid detection. The approach is based on the notion that if there is more than one 
watermarked component, the accumulated Fourier magnitude should contain a grid signal 
corresponding to each watermark. The presence of multiple grids generates multiple 
strong peaks both at the log-polar correlation and Fourier magnitude correlation steps in 
the process. Each peak corresponds to the rotation and scale of the corresponding grid. 

The multiple grids approach can include certain of the steps detailed in Figs. 2 
and 3, which include the following: 
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• Segment the image into blocks; 

• A Ppty a non-linear prefilter that estimates the image data (from the image+watermark 
input data) and subtracts its from the input data. (Estimation can be performed using 
the host signal estimation technique disclosed in application 09/278,049.) 

• Compute the detection value corresponding to each of the N (e.g., 2-16) candidates 
in the Fourier Magnitude Correlation (FMC) step. 

• Select the M candidates that surpass the two-step rejection criterion. 

• Try refining scale/rotation on these M candidates to obtain refined rotation and scale 
as well as differential scale and shear (i.e., a 4D refinement vector). 

• Loop through the M 4D vector sets and perform translation determination and read 
(decode) operations for each, using all available blocks. 

• Buffer the decoded watermark information from each of the M candidates (i.e., read 
or not read, if read what type of watermark, and so on). 

• Either return this information to the calling program, or select the watermark to return 
to the calling program based upon a pre-determined priority ranking of the various 
possible watermarks. 

This approach works as long as there is sufficient signal strength corresponding to 
each potential grid in the accumulated Fourier magnitude. Also note that this approach 
assumes that all the watermarks in the image have the same grid. Experiments using this 
approach have yielded promising results and shown that this approach is feasible. 

Such an approach has various advantages, including: 

• It enables detection of multiple watermarks even if the multiple watermarked 
documents are overlapped or in close proximity to each other. 

• It does not affect the performance (speed) of the grid detection stage. 

• Most of the methodology parallels prior art techniques; the sequence of operations 
after the grid detection stage need to be repeated for each detected grid. 

• It is possible to extend this approach to situations where there are more than two or 
three watermarked components in an image. 
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A few shortcomings persist: 

• If the multiple watermarks are weak, it may be difficult to detect them in the 
accumulated Fourier magnitude. 

• One challenge in this approach arises if the multiple grids have almost the same 
rotation and scale. This limitation arises because the peak finding algorithm cannot 
resolve closely located peaks in the GMF correlation plane. 

The two-step rejection criteria noted above is more particularly detailed in 
application 09/526,982. 

An exemplary grid signal is one with the following characteristics: 

• It comprises a collection of impulse or delta functions in the Fourier magnitude 
domain. 

• The impulse functions have pseudo random phase (i.e. the phase is random, yet 
known so that translation (its X and Y origin) of the watermark can be computed by 
correlating the phase information of the calibration signal with the watermarked 
signal) 

• The impulse functions are typically distributed in the mid-frequency range so as to 
survive distortion yet not be perceptible 

In other embodiments, different grid signals can be used in differently- 
watermarked excerpts of the content. Figs. 2 and 3 more particularly detail a detection 
process useful in this context. 

As before, the image is segmented into blocks, pre-filtered, the converted into the 
Fourier domain. The Fourier representation for all the component blocks are 
accumulated, filtered, and remapped into the log-polar domain. 

In contrast to the multiple-same grid context, the multiple-different grid context 
process proceeds by correlating the log-polar representation obtained above, with a log- 
polar remapping of the Fourier magnitude representation of each of the component grid 
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signals, summed together (a summed-grid template). This correlation yields several 
peaks, each indicating a candidate scale/rotation state of one of the component 
watermarked elements. But the peaks do not indicate the particular grid signals with 
which they correlated, since the correlation was based on a summed-grid template. 
Accordingly, the method proceeds by checking each discerned scale/rotation state 
(correlation peak) of the data against the Fourier magnitude spectrum of the different grid 
signals, to determine which grid signal should be used in decoding a given scale/rotation 
state of the data. Once this association between scale/rotation states of data, and 
applicable grid signal, has been determined, decoding of each can proceed as above (and 
in the cited applications). 

Of course, for any single grid signal, there may be several objects represented in 
the data set - each with a different scale or rotation. 

In accordance with yet another aspect of the invention, the impulse functions can 
be modulated to carry auxiliary information as follows: 

Encode: 

a. create message (e.g., binary bit string) 

b. error correction encode and/or spread spectrum modulate the string 

c. map elements of resulting message signal to fourier magnitude impulse 
function locations 

d. encode 1 as positive impulse function and encode 0 as negative impulse 
function (or other predetermined relation) 

Detect and decode message: 

a. detect impulses to determine whether a watermark is present; 

b. if detected, then go back and check for the presence at predetermined Fourier 
Magnitude impulse function locations; 

c. perform inverse of spread spectrum modulation and error correction coding to 
recover original message 
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Note that there are many possible applications: The calibration signal can carry 
protocol information to tell the decoder how to interpret the message payload of another 
watermark signal. 

In systems in which the "grid" signal conveys the message, there is no need for 
5 separate "grid" and "message" signals. 

For additional information on the use of the grid signal to convey payload 
information, see pending application 09/618,948. 

Having described and illustrated the principles of our inventive work with 
reference to specific embodiments, it will be recognized that the principles thereof can be 
10 implemented in other, different, forms. 

For example, while the disclosure focused on image data, the same techniques are 
applicable in other watermarking contexts, including audio and video. 

Moreover, while the invention was illustrated in the context of the present 
assignee's preferred forms of watermarking, it should be recognized that the invention's 
15 applicability is not so limited. For example, such techniques also find utility in 
combination with the teachings of watermarking patents 5,949,055, 6,044,182, etc. 

It should be recognized that the particular combinations of elements and features 
in the above-detailed embodiments are exemplary only; the interchanging and 
substitution of these teachings with other teachings in this and the incorporated-by- 
20 reference patents/applications are also contemplated. 

As is familiar to those skilled in the arts, all of the foregoing methods may be 
performed using dedicated hardware components/systems for the various devices, and/or 
through use of processors programmed in accordance with firmware or software, etc. In 
the latter case the processors may each include a CPU and associated memory, together 
25 with appropriate input and output devices/facilities. The software can be resident on a 
physical storage media such as disks, and can be loaded into the processors' memory for 
execution. The software includes instructions causing the CPU to perform the various 
processes detailed above. 
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To provide a comprehensive disclosure without unduly lengthening this 
specification, the patents and applications cited above are incorporated herein by 
reference. 

In view of the wide variety of embodiments to which the principles of our 
inventive work can be applied, it should be recognized that the detailed embodiments are 
illustrative only and should not be taken as limiting the scope of the invention. Rather, 
we claim as our invention all such embodiments as may come within the scope and spirit 
of the following claims, and equivalents thereof 



